Eyeglasses-type wearable terminal, control method thereof, and control program

ABSTRACT

There is provided an eyeglasses-type wearable terminal that causes a user to more reliably recognize the position of a predetermined object, which includes an image capturing unit that captures the periphery of the user. The eyeglasses-type wearable terminal also includes a determiner that determines whether a predetermined target object is included in a video acquired by image capturing using the image capturing unit. The eyeglasses-type wearable terminal further includes a display unit that displays the position of the predetermined target object in a case in which the determiner determines that the predetermined target object is included.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/JP2016/088186 filed Dec. 21, 2016, claiming priority based on theJapanese Patent Application No. 2016-058025 filed Mar. 23, 2016.

TECHNICAL FIELD

The present invention relates to an eyeglasses-type wearable terminal, acontrol method thereof, and a control program.

BACKGROUND ART

In the above technical field, patent literature 1 discloses a techniqueof specifying a suspicious person in an image captured by a cameraprovided on an eyeglasses-type wearable terminal and outputting awarning message. Patent literature 2 discloses a technique of causing aneyeglasses-type terminal with a peripheral camera to notify the wearerof the terminal that an object is approaching.

CITATION LIST Patent Literature

Patent literature 1: Japanese Patent Laid-Open No. 2010-081480

Patent literature 2: Japanese Patent Laid-Open No. 2013-008307

SUMMARY OF THE INVENTION Technical Problem

However, by the techniques described in the above literatures, it isimpossible to know where the suspicious person or the object exists.

The present invention enables to provide a technique of solving theabove-described problem.

Solution to Problem

One example aspect of the present invention provides an eyeglasses-typewearable terminal comprising:

an image capturing unit that captures a periphery;

a determiner that determines whether a predetermined target object isincluded in a video captured by the image capturing unit; and

a display unit that displays a position of the predetermined targetobject in a case in which the determiner determines that thepredetermined target object is included.

Another example aspect of the present invention provides a controlmethod of an eyeglasses-type wearable terminal including an imagecapturing unit that captures a periphery, comprising:

determining whether a predetermined target object is included in a videocaptured by the image capturing unit; and

displaying a position of the predetermined target object in a case inwhich it is determined in the determining step that the predeterminedtarget object is included.

Still other example aspect of the present invention provides a controlprogram of an eyeglasses-type wearable terminal including an imagecapturing unit that captures a periphery, for causing a computer toexecute a method, comprising:

determining whether a predetermined target object is included in a videocaptured by the image capturing unit; and

displaying a position of the predetermined target object in a case inwhich it is determined in the determining step that the predeterminedtarget object is included.

Advantageous Effects of Invention

According to the present invention, a user can recognize the position ofa predetermined object using an eyeglasses-type wearable terminal.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing the functional arrangement of aneyeglasses-type wearable terminal according to the first exampleembodiment of the present invention;

FIG. 2A is a block diagram showing the functional arrangement of aneyeglasses-type wearable terminal according to the second exampleembodiment of the present invention;

FIG. 2B is a view showing the outer appearance of the eyeglasses-typewearable terminal according to the second example embodiment of thepresent invention;

FIG. 3A is a flowchart showing the procedure of processing of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention;

FIG. 3B is a flowchart showing the procedure of processing of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention;

FIG. 4 is a view showing an example of the display screen of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention;

FIG. 5 is a view showing an example of the display screen of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention;

FIG. 6 is a view showing an example of the display screen of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention; and

FIG. 7 is a view showing an example of the display screen of theeyeglasses-type wearable terminal according to the second exampleembodiment of the present invention.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Example embodiments of the present invention will now be described indetail with reference to the drawings. It should be noted that therelative arrangement of the components, the numerical expressions andnumerical values set forth in these example embodiments do not limit thescope of the present invention unless it is specifically statedotherwise.

[First Example Embodiment]

An eyeglasses-type wearable terminal 100 according to the first exampleembodiment of the present invention will be described with reference toFIG. 1. As shown in FIG. 1, the eyeglasses-type wearable terminal 100includes an image capturing unit 101, a determiner 102, and a displayunit 103.

The image capturing unit 101 captures at least the front of theeyeglasses-type wearable terminal 100. The determiner 102 determineswhether a predetermined target object is included in a video captured bythe image capturing unit 101.

If the determiner 102 determines that the predetermined target object isincluded, the display unit 103 displays the position of thepredetermined target object.

According to the above-described arrangement, a user can more reliablyrecognize the position of a predetermined object using theeyeglasses-type wearable terminal.

[Second Example Embodiment]

An eyeglasses-type wearable terminal 200 according to the second exampleembodiment of the present invention will be described with reference toFIGS. 2A and 2B. FIG. 2A is a block diagram showing the functionalarrangement of the eyeglasses-type wearable terminal 200 according tothis example embodiment, and FIG. 2B is a view showing the outerappearance of the eyeglasses-type wearable terminal 200.

As shown in FIG. 2A, the eyeglasses-type wearable terminal 200 includesa front camera 201, a rear camera 202, a display 203, a microphone 204,a speaker 205, a communication unit 206, an operation unit 207, a memory208, and a processor 209. These are functional components basicallyimplemented by hardware. In addition, the eyeglasses-type wearableterminal 200 includes an image generator 210, a target detector 211, adistance determiner 212, an approach determiner 213, a face identifier214, a congestion degree determiner 215, and a position/azimuth detector216. The components 210 to 216 are basically implemented by software.That is, these are functions implemented by executing program modulesstored in the memory 208 by the processor 209. The display 203 may existon each of the left and right sides. In addition, the ranges of the leftand right lenses may be determined, and it may be determined which oneof the displays should perform display.

The front camera 201 and the rear camera 202 capture the periphery ofthe eyeglasses-type wearable terminal 200. The communication unit 206can communicate with an external radio tag 220 and acquires the positioninformation of the radio tag 220 from a position detector 221. Theposition detector 221 may include a GPS (Global Positioning System)receiver and receive the absolute position of the radio tag 220 on theearth.

The operation unit 207 accepts an operation from a user who wears theeyeglasses-type wearable terminal 200. The memory 208 stores variouskinds of data in addition to various kinds of programs.

The image generator 210 generates an image to be displayed on thedisplay 203. More specifically, the image generator 210 generates animage that displays the position of a predetermined target object. Thetarget detector 211 detects a predetermined target based on a signalreceived from the radio tag 220 or determines whether a predeterminedtarget object is included in a captured image. The distance determiner212 determines the distance up to a predetermined target based on thestrength of a signal received from the radio tag 220. The approachdeterminer 213 analyzes an image captured by the front camera 201 or therear camera 202 and determines whether an abruptly approaching objectexists.

The face identifier 214 analyzes images captured by the cameras 201 and202, identifies faces in the images, and determines whether a personregistered in advance exists on the periphery. The congestion degreedeterminer 215 analyzes images captured by the cameras 201 and 202 anddetermines the congestion degree on the periphery. The position/azimuthdetector 216 detects the position and direction (azimuth) of theeyeglasses-type wearable terminal 200.

FIGS. 3A and 3B are flowcharts showing the procedure of processing ofthe eyeglasses-type wearable terminal 200. When a signal including theposition information of the radio tag 220 is received from the radio tag220 in step S301, the process advances to step S303. In step S303, theposition information of the eyeglasses-type wearable terminal 200detected by the position/azimuth detector 216 and the positioninformation of the radio tag 220 are compared, and it is determinedwhether the direction of the radio tag 220 viewed from theeyeglasses-type wearable terminal 200 falls within the display range ofthe display 203. The display range of the display 203 can be calculatedbased on the direction (azimuth) of the eyeglasses-type wearableterminal 200 detected by the position/azimuth detector 216. For example,in a case in which the azimuth of the eyeglasses-type wearable terminal200 is the due north, if the radio tag 220 is included in the range from45° east longitude to 45° west longitude when viewed from theeyeglasses-type wearable terminal 200, it is determined that thedirection of the radio tag 220 falls within the display range of thedisplay 203.

If the radio tag 220 exists within the display range of the display 203,the process advances to step S307, and the image generator 210 displaysa target name represented by the radio tag 220 and a downward arrow onthe upper side of the position of the radio tag 220. At this time, thesize of the arrow or the number of arrows is set to a size or numberaccording to the distance between the radio tag 220 and theeyeglasses-type wearable terminal 200. If the radio tag 220 transmitsinformation representing a clerk or a station worker, display asindicated by a screen 401 or a screen 402 shown in FIG. 4 is obtained.In the screen 401, “station worker” is displayed as a target namedisplay 411, and an arrow 412 is displayed as an identifier representinga position such that they are overlaid on the background seen throughthe glass display. In the screen 402, “clerk” is displayed as each oftarget name displays 421 and 422, and arrows 423 and 424 are displayedas identifiers representing positions. Since the clerk indicated by thearrow 424 is close to the eyeglasses-type wearable terminal 200, thearrow 424 is displayed to be larger than the arrow 423. The color of thearrow may be changed in accordance with the distance to the target.

If the radio tag 220 exists outside the display range of the display203, the process advances to step S309, and a target name represented bythe radio tag 220 and an arrow representing the direction (the rightside or the left side of the display range) are displayed. At this time,the size of the arrow or the number of arrows is set to a size or anumber according to the distance between the radio tag 220 and theeyeglasses-type wearable terminal 200. For example, in a screen 500shown in FIG. 5, “station worker” as a target name display 501 andarrows 502 are displayed. In this example, two arrows 502 are displayedbecause the station worker is away. If the station worker is fartherapart, three arrows are displayed. If the station worker is close, onearrow is displayed. The radio tag 220 can be attached not only to aclerk or station worker but to an arbitrary person or object. When theinformation of a user's child is stored in the radio tag 220, and theradio tag 220 is attached to a cloth of the child or the like, as shownin FIG. 6, even if a child 601 gets lost, a parent 602 who wears theeyeglasses-type wearable terminal 200 can see a screen display 603 andimmediately recognize which child exists at which position in whichdirection at which distance.

The above-described processing shown in FIG. 3A is started when arequest operation to display the location of the target object is inputfrom the user. However, depending on the attribute of information in theradio tag, a setting may be done such that the distance between theradio tag 220 and the eyeglasses-type wearable terminal 200 isperiodically calculated, and alert display (including the direction andthe distance) is performed in a case in which they are far apart by apredetermined distance or more. For example, when the radio tag isattached to the child, the parent can immediately be notified that thechild 601 is away.

Referring to FIG. 3B, in step S311, the approach determiner 213 analyzesthe images captured by the cameras 201 and 202 and determines whether anapproaching object that is moving toward the eyeglasses-type wearableterminal 200 exists. If an approaching object exists, the processadvances from step S313 to step S315, and the congestion degreedeterminer 215 determines the congestion degree around the user whowears the eyeglasses-type wearable terminal 200. In addition, apredetermined speed (threshold) set in accordance with the determinedcongestion degree and the approaching speed of the approaching objectare compared. In congestion, a possibility that a person is erroneouslydetected as an approaching person becomes high. Hence, the predeterminedspeed is set higher to reduce detection errors in association withpassing pedestrians. During a high-speed movement (during driving of avehicle, in a train, or the like), collision detection may be turnedoff. As for the congestion degree, faces in an image are detected. Ifthe number of detected faces is a predetermined number or more, it maybe determined that there is congestion. In addition, the congestiondegree is an index value representing the degree of the number ofpersons existing in the monitoring place, and may be represented by thenumber of persons, the density, or a value obtained by calculation usingat least one of the number of persons and the density. For example, anarrangement that calculates the index of the congestion degree based onthe number of detected faces can be employed. Additionally, as themethod of representing the congestion degree as a numerical value, it ispossible to detect a crowd (a cluster of persons) and accuratelyestimate the number of persons in a monitoring image using the number ofdetected people in the crowd.

If the approaching speed of the object is a predetermined speed or more,the process advances to step S319, and the image generator 210 displaysthe position (the direction and the distance) of the approaching object.FIG. 7 shows a screen 700 in a case in which alert display is performedfor approaching persons. If an approaching person exists, as shown inthe screen 700, the image generator 210 displays a warning “attention toapproach!” as a target name display 701, and displays an arrow 702 as anidentifier representing a position in a size according to the distanceto the person. These images are displayed such that they are overlaid onthe background seen through the glass display. In addition, if the userhits the approaching person (the acceleration sensor detects anacceleration of a predetermined value or more), videos captured by thecameras 201 and 202 are recorded.

[Other Example Embodiments]

While the invention has been particularly shown and described withreference to example embodiments thereof, the invention is not limitedto these example embodiments. It will be understood by those of ordinaryskill in the art that various changes in form and details may be madetherein without departing from the spirit and scope of the presentinvention as defined by the claims.

The present invention is applicable to a system including a plurality ofdevices or a single apparatus. The present invention is also applicableeven when an information processing program for implementing thefunctions of example embodiments is supplied to the system or apparatusdirectly or from a remote site. Hence, the present invention alsoincorporates the program installed in a computer to implement thefunctions of the present invention by the processor of the computer, amedium storing the program, and a WWW (World Wide Web) server thatcauses a user to download the program. Especially, the present inventionincorporates at least a non-transitory computer readable medium storinga program that causes a computer to execute processing steps included inthe above-described example embodiments.

[Other Expressions of Embodiments]

Some or all of the above-described embodiments can also be described asin the following supplementary notes but are not limited to thefollowings.

(Supplementary Note 1)

There is provided an eyeglasses-type wearable terminal comprising:

an image capturing unit that captures a periphery;

a determiner that determines whether a predetermined target object isincluded in a video captured by the image capturing unit; and

a display unit that displays a position of the predetermined targetobject in a case in which the determiner determines that thepredetermined target object is included.

(Supplementary Note 2)

There is provided the eyeglasses-type wearable terminal according tosupplementary note 1, wherein in a case in which the predeterminedtarget object exists outside a display range of the eyeglasses-typewearable terminal, the display unit displays an arrow representing adirection in which the predetermined target object exists.

(Supplementary Note 3)

There is provided the eyeglasses-type wearable terminal according tosupplementary note 2, wherein the display unit changes one of a size anda color of the arrow based on a distance up to the predetermined targetobject.

(Supplementary Note 4)

There is provided the eyeglasses-type wearable terminal according to anyone of supplementary notes 1 to 3, wherein the determiner receives asignal from a radio tag provided on the predetermined target object andperforms the determination based on the signal.

(Supplementary Note 5)

There is provided the eyeglasses-type wearable terminal according tosupplementary note 4, wherein the signal includes position informationof the predetermined target object.

(Supplementary Note 6)

There is provided the eyeglasses-type wearable terminal according to anyone of supplementary notes 1 and 5, wherein the image capturing unitcaptures a front and a rear of the eyeglasses-type wearable terminal,the determiner determines whether an object approaching theeyeglasses-type wearable terminal exists in the video, and

the display unit displays a position of the object.

(Supplementary Note 7)

There is provided the eyeglasses-type wearable terminal according to anyone of supplementary notes 1 to 6, wherein the determiner detects anobject included in the video, which approaches the eyeglasses-typewearable terminal at a speed not less than a predetermined threshold.

(Supplementary Note 8)

There is provided the eyeglasses-type wearable terminal according tosupplementary note 7, wherein the determiner determines a congestiondegree on the periphery, and if the congestion degree is high, raisesthe predetermined threshold.

(Supplementary Note 9)

There is provided the eyeglasses-type wearable terminal according to anyone of supplementary notes 1 to 5, wherein the image capturing unitcaptures a front and a rear of the eyeglasses-type wearable terminal,

the determiner determines whether a person registered in advance isincluded in the video, and

the display unit displays a position of the person registered inadvance.

(Supplementary Note 10)

There is provided a control method of an eyeglasses-type wearableterminal including an image capturing unit that captures a periphery,comprising:

determining whether a predetermined target object is included in a videocaptured by the image capturing unit; and

displaying a position of the predetermined target object in a case inwhich it is determined in the determining that the predetermined targetobject is included.

(Supplementary Note 11)

There is provided a control program of an eyeglasses-type wearableterminal including an image capturing unit that captures a periphery,for causing a computer to execute a method, comprising:

determining whether a predetermined target object is included in a videocaptured by the image capturing unit; and

displaying a position of the predetermined target object in a case inwhich it is determined in the determining that the predetermined targetobject is included.

The invention claimed is:
 1. An eyeglasses-type wearable terminalcomprising: a camera configured to capture a periphery; at least oneprocessor configured to determine whether a predetermined target objectis included in a video captured by the camera; and a display unitconfigured to display a position of the predetermined target object in acase in which the at least one processor determines that thepredetermined target object is included, wherein the display unit isconfigured to, in a case in which the predetermined target object existsoutside a display range of the eyeglasses-type wearable terminal,display an arrow representing a direction in which the predeterminedtarget object exists, and wherein the display unit is configured tochange at least one of a number, a size and a color of the arrow basedon a distance up to the predetermined target object.
 2. Theeyeglasses-type wearable terminal according to claim 1, wherein the atleast one processor is configured to receive a signal from a radio tagprovided on the predetermined target object and to perform thedetermination based on the signal.
 3. The eyeglasses-type wearableterminal according to claim 2, wherein the signal includes positioninformation of the predetermined target object.
 4. The eyeglasses-typewearable terminal according to claim 1, wherein the camera is configuredto capture a front and a rear of the eyeglasses-type wearable terminal,wherein the at least one processor is configured to determine whether anobject approaching the eyeglasses-type wearable terminal exists in thevideo, and wherein the display unit is configured to display a positionof the object.
 5. The eyeglasses-type wearable terminal according toclaim 1, wherein the at least one processor is configured to detect anobject included in the video, which approaches the eyeglasses-typewearable terminal at a speed not less than a predetermined threshold. 6.An eyeglasses-type wearable terminal comprising: a camera configured tocapture a periphery; at least one processor configured to determinewhether a predetermined target object is included in a video captured bythe camera; and a display unit configured to display a position of thepredetermined target object in a case in which the at least oneprocessor determines that the predetermined target object is included,wherein the at least one processor is configured to detect an objectincluded in the video, which approaches the eyeglasses-type wearableterminal at a speed not less than a predetermined threshold, and whereinthe at least one processor is configured to determine the predeterminedthreshold according to a congestion degree on the periphery.
 7. Theeyeglasses-type wearable terminal according to claim 1, wherein thecamera is configured to capture a front and a rear of theeyeglasses-type wearable terminal, wherein the at least one processor isconfigured to determine whether a person registered in advance isincluded in the video, and wherein the display unit is configured todisplay a position of the person registered in advance.
 8. A controlmethod of an eyeglasses-type wearable terminal including a camera thatcaptures a periphery, the control method comprising: determining whethera predetermined target object is included in a video captured by thecamera; and displaying a position of the predetermined target object ina case in which it is determined in the determining that thepredetermined target object is included, wherein in a case in which thepredetermined target object exists outside a display range of theeyeglasses-type wearable terminal, an arrow representing a direction inwhich the predetermined target object exists is displayed, and whereinat least one of a number, a size and a color of the arrow is changedbased on a distance up to the predetermined target object.
 9. Anon-transitory computer readable medium storing a program which, ifexecuted, controls an eyeglasses-type wearable terminal including acamera that captures a periphery, by causing a computer to execute amethod comprising: determining whether a predetermined target object isincluded in a video captured by the camera; and displaying a position ofthe predetermined target object in a case in which it is determined inthe determining that the predetermined target object is included,wherein in a case in which the predetermined target object existsoutside a display range of the eyeglasses-type wearable terminal, anarrow representing a direction in which the predetermined target objectexists is displayed, and wherein at least one of a number, a size and acolor of the arrow is changed based on a distance up to thepredetermined target object.
 10. A control method of an eyeglasses-typewearable terminal including a camera that captures a periphery, thecontrol method comprising: determining whether a predetermined targetobject is included in a video captured by the camera; and displaying aposition of the predetermined target object in a case in which thedetermining determines that the predetermined target object is included,wherein an object included in the video, which approaches theeyeglasses-type wearable terminal at a speed not less than apredetermined threshold is detected, and wherein the predeterminedthreshold is determined according to a congestion degree on theperiphery.
 11. A nontransitory computer readable medium storing aprogram which, if executed, controls an eyeglasses-type wearableterminal including a camera that captures a periphery, by causing acomputer to execute a method comprising: determining whether apredetermined target object is included in a video captured by thecamera; and displaying a position of the predetermined target object ina case in which the determining determines that the predetermined targetobject is included, wherein an object included in the video, whichapproaches the eyeglasses-type wearable terminal at a speed not lessthan a predetermined threshold is detected, and wherein thepredetermined threshold is determined according to a congestion degreeon the periphery.